A Bayesian Recognitional Decision Model

In this paper, I describe the Bayesian Recognitional Decision Model (BRDM), a bayesian implementation of the recognition-primed decision model (Klein, 1993) based primarily on models of episodic recognition memory (Shiffrin & Steyvers, 1997; Mueller & Shiffrin, 2006). The proposed model accounts for three important factors used by experts to make decisions: the evidence about a current situation, the prior base rate of event classes in the environment, and the reliability of the information reporter. The bayesian framework integrates these three aspects of information together in an optimal way, and provides a principled framework for understanding recognitional decision processes. Summary of Prior Research The recognition-primed decision model (RPD, Klein, 1989; 1993; 1998) arose from naturalistic observation of expert human decision makers. Its basis in naturalistic settings and applied research helped identify the types of decisions and situations that appear most relevant to expert decision makers in the world. These situations often involved “time pressure, uncertainty, ill-defined goals, high personal stakes, and other complexities” (Lipshitz, Klein, Orasanu, & Salas, 2003). The main insight of the RPD model is that the skill of expert decision making lies in making sense of the current situation, identifying past situations that were similar, and using the workable actions that Part of this research was presented at the Workshop on Developing and understanding Computational Models of Macrocognition, Havre de Grace, MD, June 3–4, 2008, organized by Laurel Allender and Walter Warwick. This work partly funded by the Defense Threat Reduction Agency. The author thanks Anna Grome and Beth Crandall for conducting interviews and developing qualitative models of chemical officers upon which some of the present model is based, and Gary Klein and Rich Shiffrin for developing many of the ideas this model was based upon. A BAYESIAN RECOGNITIONAL DECISION MODEL 2 were taken in the past to guide current choices. In contrast to the processes hypothesized by more classic theories of decision making, naturalistic decision making is not about balancing trade-offs between choices, estimating probabilities, or assessing the expected utility of features or options. The RPD model describes a fairly complex set of processes used by experts to make decisions. Despite the fact that the model discusses concepts such as pattern matching, recognition memory, and situation-action decision rules (all of which constitute psychological theories that have been embedded or implemented in computational and mathematical models), it is fundamentally a conceptual model of decision making, without a computational or mathematical implementation. Yet there are aspects of the RPD theory that are similar to a number of computational models of decision making, and so these other models may be able to provide reasonable mechanistic and formal models of the RPD theory. For example, at a high level, random walk and accumulator models arising out of the stochastic process tradition (Ratcliff, 1978; Busemeyer & Townsend, 1993; Usher & McClelland, 2001) offer explicit mechanisms for dealing with time-pressured decision making. These models accumulate evidence for different options in microsteps, and come to a decision when evidence for an option passes some boundary, or based on a accumulated evidence at some stopping point. To the extent that RPD was born out of situations requiring immediate time-sensitive action, these models may offer insights into a similar domain. In fact, they offer natural accounts of satisficing (making a decision that is good enough) and that are central to the RPD model. Furthermore some of these models have been closely connected to models of memory retrieval, which is also central to the RPD model. For example, Ratcliff’s seminal 1978 paper in which he introduced the diffusion model was even entitled “A theory of memory retrieval.” (see also Shiffrin, Ratcliff, & Clark, 1990, and Ratcliff, Clark & Shiffrin, 1990). However, these stochastic models typically require multiple well-defined choice alternatives to be available at the outset of the process. In contrast, the RPD model needs only to retrieve a single option to begin evaluating it, and this option is often good enough so that it does not need to be compared to alternatives. In the stochastic modeling tradition, many of the processes described by the RPD model have been lumped into terms such as “option generation” (Johnson & Raab, 2003; Raab & Johnson, 2007). This philosophical stance essentially treats option generation as a pre-processing step along the way to decision making. This approach makes sense for theories based on empirical paradigms in which research subjects are given carefully-crafted well-defined options to choose among in laboratory settings. However, even when option generation is considered, the focus remains on choice among options, rather than evaluation of individual options. As an example of this focus, even though typical random walk models of decision making (e.g., Ratcliff, 1979) require the comparison of two and only two options, some alternatives (e.g., Usher & McClellend, 2001) have developed alternative architectures to get around this limitation. However, they have typically done so by increasing the comparison process to 3, 4, or more options, rather than decreasing it to evaluating the goodness of a single option. For the RPD model, option generation is the tail that wags the dog of decision making. The skills that make experts good at their decisions is their ability to generate good options—perhaps so much so that there is often no need to generate multiple options and evaluate them against one another. Consequently, these stochastic models of decision making remain rooted in a tradition of comparison and choice among options, and may not be well suited for developing a computational model of expert decisions. A more fruitful area of computational modeling relevant to RPD comes from models of A BAYESIAN RECOGNITIONAL DECISION MODEL 3 episodic and semantic memory. Because the core of the RPD model lies in identifying past situations that are similar to the current one (given goals and context), this is essentially a question of memory retrieval. A number of computational models have explored these notions in the context of episodic memory, memory for lists of words or pictures, categorization, priming, lexical decision, and other related phenomena. Two prominent models include MINERVA 2 (Hintzmann, 1984), and REM (Shiffrin & Steyvers, 1997). The core assumptions of these models is that memories for events take on feature-based representations, and are stored as individual traces. MINERVA 2 stores past memory entirely as episodes, and uses similarity matching rules to determine what is retrieved or recalled. REM allows for long-term memory prototypes as well as individual episodic traces to exist, and frames the memory retrieval process as a bayesian decision problem, enabling evidence, uncertainty, and prior probabilities to be incorporated in principled (and even optimal) ways. This bayesian framing was at least partly inspired by Anderson’s (1990) development of the ACT-R model, another model that embeds theories of memory retrieval with close connections to decision making. Following Hintzman’s (1988) extension of MINERVA 2 to a decision realm, Dougherty and colleagues (e.g., Dougherty et al., 1999, Dougherty, 2001) developed the MINERVA-DM decisionmaking model rooted in episodic memory theory. MINERVA-DM resembles aspects of the RPD model in that past experiences are used as a basis for decision making. However, the goal of that model was to account for human errors in frequency, likelihood, and probability estimation. This motivation places the model outside the domain of RPD, as it focuses on tasks that are not relevant to much of expert decision making. Yet the essence of the model was to explore connections between decision making and memory, which is a critical insight of the RPD model. Another model that adapted MINERVA 2 to more closely implement aspects of the RPD theory was developed by Warwick and colleagues (Warwick & Hutton, 2001; see also Warwick, in the current issue). This model resembles Hintzmann (1988) and Dougherty’s (2001) models, but focuses more on naturalistic decisions. It constitutes an important component of the RPD model, as it provides a formal account of the pattern matching process central to recognitional decision making. More recently, Yen and colleagues (Fan, Sun, McNees, & Yen, 2005; Fan, Sun, Sun, McNees, and Yen, 2006; Ji, Massanari, Ager, Yen, Miller, & Ying, 2007) have reported on developments of a multi-agent modeling system called R-CAST. R-CAST is not a model of human cognitive processes per se, it is rather an artificially intelligent multi-agent system in which agents have RPD decision making capabilities. It is intended to model group behavior, and to serve as reasonable team-mates and decision aids. Thus, new computational models of the RPD process have a number of prior models to consider. These include models arising from traditional decision theory, from cognitive theory on memory and knowledge, and even from computational models of RPD coming from applied settings. In this paper, I will advocate and describe the use of a bayesian modeling framework within which to develop a computational RPD model. The bayesian framework offers a principled way of developing a model, and it provides natural ways to capture three important aspects of expert decision making we have identified in our applied research: the decision maker’s use of evidence about the current state of the world; the use of background information about the likely states of the world, and an assessment about the reliability of the person or system providing that evidence. This Bayesian Recognitional Decision Model (BRDM) is described next.1 The RPD model described by Klein (1993; 1998) is fairly extensive and covers processes such as pattern matching, A BAYESIAN RECOGNITIONAL DECISION MODEL 4 A Bayesian Recognitional Decision Model (BRDM) Bayesian models of psychological processes offer principled approaches to understanding how evidence is used in various decision processes. These include decisions about episodic memory (REM, Shiffrin & Steyvers, 1997; BCDMEM, Dennis & Humphreys, 2001), semantic knowledge (REM-II, Mueller & Shiffrin, 2006), perceptual judgment (ROUSE, Huber et al., 2001), and word recognition (the bayesian reader, Norris, 2006). Such models are not optimal in the sense that they are always correct, but rather they allow for optimal performance under specified limitations (such as memory errors or biases). These models suggest methods for implementing a bayesian computational model of recognitional decision making. An important feature of these models is how they use evidence and combine it with background knowledge of how likely different events are. This is the essence of Bayes rule, which optimally combines present evidence with base rate priors. In addition, the bayesian framework allows one to specify directly a decision maker’s model of how the evidence was generated, and so things like the perceived reliability of a report can also be incorporated optimally. The bayesian frame of the BRDM is both a theoretical assumption and a practical modeling approach. Many computational decision models offer a number of free parameters with which to explain data. These can represent processes such as evidence sampling, decision thresholds, intrinsic noise, etc. Often, different sources of noise can masquerade as one another, and so the true source of effects can be difficult to determine (cf. Mueller & Weidemann, 2008). The bayesian approach makes a default assumption that evidence is used optimally, and so the interesting processes lie elsewhere. This is reasonable as a default because there are many ways to be sub-optimal, but only one way to be optimal. Many of these optimality assumptions can be relaxed if the circumstances warrant. Thus, although naturalistic decision making research was in part a reaction against classic models of optimal decision making (cf. Lipshitz et al., 2003), the BRDM does not violate these basic insights. Rather, it helps focus theory on identifying information and knowledge used by experts to make decisions. Empirical Context of Model Development This research is based partially on interviews with Army and Air Force officers responsible for Chemical/Biological (CB) defense. Their primary role is to evaluate sensor and human reports regarding the presence of chemical or biological threats and attacks. Because the incidence of actual events is very rare (and an officer will often endure an entire career without witnessing a true positive event), this might be considered an ultra-vigilance task. I will not provide a detailed account of their decision making process, but summarize the three types of information these officers used to determine the validity of a threat. These include: • Background Information • The cues in the reports • The reliability of the source of the reports Our interviews suggested that once a situation is identified, a fairly scripted sequence of events is executed for addressing that situation. This script is often part of doctrine and the local techniques, tactics, and procedures, and indeed is often pre-planned. These scripts are wellaction selection, analogy, and mental simulation. The aspect of the model covered in this paper is focused on the process of recognizing relevant past events, and I use the term “Recognitional Decision” to highlight that the fact that this model does not incorporate many of the more complex and interesting aspects of behavior described by the RPD model. A BAYESIAN RECOGNITIONAL DECISION MODEL 5 Table 1: Example feature-based representations of events and reports about events. Probabilistic Event Prototype .9 .01 .2 .1 .95 Binary Event Prototype 1 0 0 0 1 Feature Base Rate .8 .3 .2 .1 .6 Report about event 1 0 0 1 1 practiced and exercised, and although they are unlikely to fit a situation exactly, their execution is routinized once a situation is identified. Our interviews suggest that an attack is typically treated as a true chemical attack, until a convincing level of evidence can be established to eliminate the possibility. Next, I will describe the mechanisms by which the BRDM identifies a situation based on background information, environmental cues, and the reliability of the reporter. Model of the Environment For the BRDM to operate, we must first make assumptions about how events occur in the environment and are reported to the decision maker. First, we assume that there are a number of classifiable events that might occur. In the chemical/biological weapons protection domain, these event classes might correspond to things like conventional missile attacks, false alarms from environmental dust, nerve agents delivered through ballistic missiles, and so on. These event classes have typical sets of features associated with them. For the example simulations in the next section, we will assume that each event class has a binary feature prototype, where each feature either present or absent. In fact, this is not necessary, and environmental events can be defined probabilistically (such as on the first row of Table 1). This distinction amounts to defining an object (e.g., a chair) by its most typical features (four legs, a seat, armrests, and a back), versus defining a chair by the probability of each feature occurring in chairs you have experienced (four legs 75% of the time, a seat 94% of the time, arm rests 52% of the time, and a back 85% of the time). For any particular event, we assume that these features are either present or absent: a particular chair either has armrests or it does not. An example of this is depicted in the second row of Table 1. These events may have different a priori base rates of occurring in the environment. Often, naturally-occurring classes happen with long-tail distributions, such that a few very common classes are very frequent, while many events happen rarely (see Zipf, 1954). We will assume that this is accurate for our domain, in order to assess the effect that differences in prior probability can have on recognitional decision making. In any specific situation, the state of the environment will be reported to the BRDM by a simulated sensor (either a human reporter or a networked device). The properties by which a sensor reports the state of the environment is captured in a probabilistic model, which the decision model later uses as a generative model to account for the observed data. In the simulation below, the sensor reports environmental features with a fixed level of accuracy r. When the sensor does not report accurately, the feature value it reports depends upon the base rate of that feature in the environment (an example of which is shown in the third row of Table 1). More formally, if B(r) is the result of a Bernoulli trial such that B(r) = 1 with probability r and 0 otherwise, and br ∈ {0, 1} is a particular result of one such trial, then the generating model for a sensor report (S) can be written: Si = br ×B(Gi) + (1 − br) ×B(Fi) (1) A BAYESIAN RECOGNITIONAL DECISION MODEL 6 Here, r is the reliability of a sensor, Gi is the ground truth of for feature i, Fi is the background frequency distribution of each feature i, and Ri is the resulting features reported to the decision model. So, for an event and a feature base rate shown in Table 1, one might observe a report like the one seen in the fourth row of Table 1. There, the event prototype was mostly correct; its one incorrect feature occurred for the fourth feature, which was a likely error to come from the feature base rate. Model of the decision maker learning For the BRDM to operate, it needs to learn about the environment it is operating in. In reality, this history comes from a number of sources: from textbook resources, from anecdotes and wisdom passed down by other officers, and from experienced training exercises and actual events. To simulate this learning process, we expose the model to a large number of exemplars of each of the relevant events. The relative probabilities of event classes are specified with a base rate distribution following Zipf’s law, and reports generated with noise introduced by a probabilistic model of information reporting, which we describe next. The BRDM learns two primary things about these events: (1) a typical featural prototype for each event class; and (2) a base rate for each event in the environment. Although individual reports provide an incomplete and unreliable view of the world, the model learns prototypes by accumulating an average representation of the observed exemplars. That is, if three exemplars of an event class had features [1 1 1 0 0] and two had features [1 1 0 0 0], its composite representation would be [1 1 .6 0 0]. Thus, an event class prototype becomes the average feature distribution of events having that label. The environmental model generates information in a noisy fashion (sometimes erroneously sampling from the base rate of a feature instead of the ground truth), and so the representations for low-frequency states become contaminated by features from high-frequency states (see Figure 1). In Figure 1, each row represents an event class, and each cell shows the relative strength of a feature for that event, with darker cells indicating greater importance. For five states with 20 features, the true environmental prototypes are shown in the top panel. After 10,000 reports, sampled according to a Zipf’s law distribution and generative model described earlier, the model’s event prototypes resemble the original environmental prototypes, but the high frequency events (toward the bottom) dominate the base rate and so features from those high-frequency events bleed into the low-frequency events because of the error generating process. The model also learns the base rate of different threats in the environment. In specific contexts and situations (domestic versus deployed; war-time versus peace-time; and depending upon the capabilities of the enemy) the officer will have a fairly clear picture of which threats are most likely, attained via coordination with their intelligence resources and other officers in the region. In practice, much of the officer’s daily work is actually comprised of learning how likely these different threats are. In the model, this knowledge is captured by accumulating an empirical event counter that provides a simple frequency distribution of experienced events. Clearly, if this base rate is inferred from other sources, behavior may produce biases such as depicted by the representativeness heuristic (Tversky & Kahneman, 1974). Model of the recognitional decision process Once a reasonable representation for a set of events has been learned, the model is able to classify new events based on its past experience. It receives a report and classifies the event by A BAYESIAN RECOGNITIONAL DECISION MODEL 7 Figure 1. Features for five hypothesized states of the world (top panel), and the corresponding representations learned by the model through experience with those situations. Each row depicts an event class, and each cell depicts the relative importance of each feature, with darker cells indicating more higher probabilities. 5 10 15 20 1 2 3 4 5 Ground truth features Feature number W or ld S ta te 5 10 15 20 1 2 3 4 5 Learned features Feature number H yp ot he si s identifying which of its memory prototypes was most likely to have generated the experienced event. To do so, the model weighs the evidence inherent in the report with the background probability of such a threat, and information regarding whether the reporter is reliable. The features of that message are compared in parallel to the relevant prototypes in long-term memory, and for each, a likelihood is computed, which integrates the reliability of the sensor, the current information, and the background knowledge about possible threats. This is done via a mental model of the reporting process, which corresponds to the model’s beliefs about the generating process for that report. We assume that the basic form of this mental model is accurate, although the actual parameters (e.g., base rate, reliability, feature strengths) might not be accurate. The equation for the the posterior decision likelihood of the model is: λik = (r̂ × Ĝik + (1 − r̂) × F̂i) F̂i (2) A BAYESIAN RECOGNITIONAL DECISION MODEL 8